Data management for production quality deep learning models: Challenges and solutions

Deep learning (DL) based software systems are difficult to develop and maintain in industrial settings due to several challenges. Data management is one of the most prominent challenges which complicates DL in industrial deployments. DL models are data-hungry and require high-quality data. Therefore, the volume, variety, velocity, and quality of data cannot be compromised. This study aims to explore the data management challenges encountered by practitioners developing systems with DL components, identify the potential solutions from the literature and validate the solutions through a multiple case study. We identified 20 data management challenges experienced by DL practitioners through a multiple interpretive case study. Further, we identified 48 articles through a systematic literature review that discuss the solutions for the data management challenges. With the second round of multiple case study, we show that many of these solutions have limitations and are not used in practice due to a combination of four factors: high cost, lack of skill-set and infrastructure, inability to solve the problem completely, and incompatibility with certain DL use cases. Thus, data management for data-intensive DL models in production is complicated. Although the DL technology has achieved very promising results, there is still a significant need for further research in the field of data management to build high-quality datasets and streams that can be used for building production-ready DL systems. Furthermore, we have classified the data management challenges into four categories based on the availability of the solutions.(c) 2022 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)

Two-stage offshoring: an investigation of the Irish bridge

Two-stage offshoring: an investigation of the Irish bridg

Going digital: Disruption and transformation in software-intensive embedded systems ecosystems

Digitalization is transforming industry to an extent that we have only seen the beginnings of. Across domains, companies experience rapid changes to their existing practices due to new technologies and new entrants that current businesses. While digitalization brings endless opportunities, it comes with challenges that require companies to strategically engage with partners in their surrounding ecosystems. In this paper, we study how companies in the embedded systems domain experience the process of transitioning from product-based companies to businesses where software, data, and artificial intelligence (AI) play an increasingly important role. To manage this, these companies need to evolve their existing ecosystems while at the same time create new ecosystems around new technologies. This involves maintaining existing technologies such as mechanics and electronics while at the same time expanding these with software, data, and AI. We provide a strategic decision framework that helps software-intensive embedded systems companies to successfully navigate the digital transformation. We do this in two steps. First, we present three models that provide the technical content of the strategic decision framework. Second, we provide an overview of the strategic alternatives that incumbents and new entrants have available when existing technologies are commoditizing and new technologies are introduced

Improving packaged software through online community knowledge

Packaged software development (PSD) is largely a knowledgeintense activity. Thus, it depends on the organizational capability of developing and combining market and technical knowledge into timely and competitive software products. Given customers’ situated knowledge of the software, software firms increasingly seek new ways to involve customers in their software development activities. As highlighted in the literature, one path for doing this is to use online communities. However, there exists little empirical research that examines the role that communities can play in the commercial endeavor of PSD. To address this omission, this paper examines the benefits and limits of online community use in PSD as it unfolds at the intersection between commercial software firm practices and voluntary community participation. On the basis of this examination, the paper presents implications for both research and practice

Virtual community use for packaged software maintenance

In this study, we investigated the use of virtual communities for involving distributed customers in the maintenance of packaged software. On the basis of an empirical study, we suggest that virtual communities can be usefully leveraged for corrective, adaptive, and perfective software maintenance. Specifically, the virtual community allowed for quick discovery of bugs and a rich interaction between developers and customers in the categories of corrective and adaptive software maintenance. However, although contributing also to the perfective category of software maintenance, this was the category in which several customer suggestions for modification were actually ignored by the developers. This implies that community use is indeed beneficial for maintenance related to coding and design errors as well as for maintenance of an adaptive character. However, it has limitations when associated with major changes such as software functionality addition or modification as those experienced in the category of perfective maintenance

Two-stage offshoring: an investigation of the Irish bridge

peer-reviewedThis paper investigates two-stage offshoring as experienced by the Irish sites of two large global companies, headquartered in the United States, with significant software development operations. As part of these companies, the Irish sites act as a bridge in their offshoring arrangements: While the U.S. sites offshore work to Ireland, the Irish sites offshore work further to India and, hence, have experience of being both customer and vendor in two-stage offshore sourcing relationships. Using a framework derived from relational exchange theory (RET), we conducted multiple case study research to investigate and develop an initial theoretical model of the implementation of this two-stage offshoring bridge model. Our study shows that while both companies act as bridges in two-stage offshoring arrangements, their approaches differ in relation to (1) team integration, (2) organizational level implementation, and (3) site hierarchy. Although, there are opportunities afforded by the bridge model at present, the extent to which these opportunities will be viable into the future is open to question. As revealed in our study, temporal location seems to favor a bridge location such as Ireland, certainly with United States--Asian partners. However, location alone will not be enough to maintain position in future two-stage offshoring arrangements. Furthermore, our research supports the view that offshoring tends to progress through a staged sequence of progressively lower cost destinations. Such a development suggests that two-stage offshoring, as described in this paper, will eventually become what we would term multistage offshoring

Large-scale machine learning systems in real-world industrial settings: A review of challenges and solutions

Background : Developing and maintaining large scale machine learning (ML) based software systems in an in-dustrial setting is challenging. There are no well-established development guidelines, but the literature contains reports on how companies develop and maintain deployed ML-based software systems. Objective : This study aims to survey the literature related to development and maintenance of large scale ML -based systems in industrial settings in order to provide a synthesis of the challenges that practitioners face. In addition, we identify solutions used to address some of these challenges. Method : A systematic literature review was conducted and we identified 72 papers related to development and maintenance of large scale ML-based software systems in industrial settings. The selected articles were qualita-tively analyzed by extracting challenges and solutions. The challenges and solutions were thematically synthe-sized into four quality attributes: adaptability, scalability, safety and privacy. The analysis was done in relation to ML workflow, i.e. data acquisition, training, evaluation, and deployment. Results : We identified a total of 23 challenges and 8 solutions related to development and maintenance of large scale ML-based software systems in industrial settings including six different domains. Challenges were most often reported in relation to adaptability and scalability. Safety and privacy challenges had the least reported solutions. Conclusion : The development and maintenance on large-scale ML-based systems in industrial settings introduce new challenges specific for ML, and for the known challenges characteristic for these types of systems, require new methods in overcoming the challenges. The identified challenges highlight important concerns in ML system development practice and the lack of solutions point to directions for future research

Experimentation growth: Evolving trustworthy A/B testing capabilities in online software companies

Companies need to know how much value their ideas deliver to customers. One of the most powerful ways to accurately measure this is by conducting online controlled experiments (OCEs). To run experiments, however, companies need to develop strong experimentation practices as well as align their organization and culture to experimentation. The main objective of this paper is to demonstrate how to run OCEs at large scale using the experience of companies that succeeded in scaling. Based on case study research at Microsoft, Booking.com, Skyscanner, and Intuit, we present our main contribution-The Experiment Growth Model. This four-stage model addresses the seven critical aspects of experimentation and can help companies to transform their organizations into learning laboratories where new ideas can be tested with scientific accuracy. Ultimately, this should lead to better products and services

Experimentation growth : Evolving trustworthy A/B testing capabilities in online software companies

Companies need to know how much value their ideas deliver to customers. One of the most powerful ways to accurately measure this is by conducting online controlled experiments (OCEs). To run experiments, however, companies need to develop strong experimentation practices as well as align their organization and culture to experimentation. The main objective of this paper is to demonstrate how to run OCEs at large scale using the experience of companies that succeeded in scaling. Based on case study research at Microsoft, Booking.com, Skyscanner, and Intuit, we present our main contribution—The Experiment Growth Model. This four‐stage model addresses the seven critical aspects of experimentation and can help companies to transform their organizations into learning laboratories where new ideas can be tested with scientific accuracy. Ultimately, this should lead to better products and services

Exploring the assumed benefits of global software development

In existing global software development (GSD) literature, much focus has been on identifying the challenges that practitioners may face (such as sociocultural and temporal distance issues), while potential benefits have not been extensively analyzed. We reverse this trend by studying these potential benefits. We question whether they are well-founded assumptions and whether they are attainable in practice. This paper presents findings from a multicase study at three multi-national companies that have extensive experience in GSD. We identify the benefits mentioned in GSD literature, analyze them with regards to the companies' experiences and then conclude whether or not each benefit is being realized in practice. Our findings reveal that the realization of the assumed benefits cannot be simply taken for granted